The invention generally relates to discrete sampled image data, and more particularly to up-sampling discrete sampled checkerboard decimated image data, where an interpolated pixel element (PEL) is determined according to neighboring discrete image samples.
Processing digitally captured input images is more accurate when separate color sensors are used to measure contributions of base colors used to represent color image data. Typical base colors include Red, Green, Blue (RGB), Cyan, Magenta, Yellow (CYM), or other color bases that can be combined to represent visible colors. To capture a particular input image, sensors attuned to particular base colors are used to identify each base color""s contribution to the image. In a digital context, the sensors break the input image into discrete pixel elements (PELs), and the term xe2x80x9cresolutionxe2x80x9d indicates the number of pixels available to a sensor for receiving input image color data. Common resolutions are 640xc3x97480, 800xc3x97600, 1600xc3x971200, etc. If separate color sensors are available, then the entire resolution of one sensor is available to receive a particular color plane""s input color data, e.g., that color""s contribution to the input image.
Unfortunately, multiple sensors increase cost and complexity. The complexity arises from having to precisely split incoming light into the various base colors, so that the same pixel element of each sensor receives its appropriate portion of the incoming light defining the input image. (If the input light is incorrectly split, then color distortion and mis-registration blur results.) In addition, even if light is precisely split, the splitter unavoidably absorbs input light and reduces overall light intensity reaching each of the sensors. As the sensitivity of the individual sensors do not change with their application, the available light must be increased by a factor equal to the number of sensors used, or a loss of sensitivity to light must be accepted in the implementation. Additionally, more light is lost by subsequent color filters used to key a sensor to one of the base colors. Thus, to minimize these losses, large, heavy, optically precise and pure (e.g., more expensive) lenses are required to maximize available light for each sensor. The higher costs can make the image capture device prohibitively expensive, and the additional weight can reduce the breadth of application for such an implementation.
To reduce costs complexity, and weight, and also to better utilize the light available to sensors through inexpensive optics, capturing devices are instead being developed with a single sensor. With a single sensor, no light splitter is required, resulting in more light available to the sensor. Since there is one sensor, available sensor PELs must be assigned to receive one of the base colors, normally utilizing a regular pattern which divides the PELs among the desired color planes. Thus, for each base color, there will be gaps in the received color data for sensor pixels that have been assigned to measure a different base color. To compensate for these gaps, received image data is interpolated, or up-sampled, to guess missing image intensity values.
One well-known regular pattern decimation method is the BAYER technique (U.S. Pat. No. 3,971,065). This technique results in one color plane that is arranged in a checkerboard pattern, using 50% of the PELs, with the other two planes linearly decimated and using 25% of the PELs each. In U.S. Pat. No. 4,630,307, Cok then: teaches a pattern recognition method for up sampling the resultant data. These patents assume an RGB color scheme and teach sampling an image with the checker board, PELs dedicated to receiving green color intensities, while red and blue are assigned: the remaining linearly sampled sensor PELs (e.g., the sensor pixels define a grid having receptors keyed as GRGB . . . both horizontally and vertically from the leading G (see:
FIG. 6 of 3,971,065). The ratio accentuating green values corresponds to research indicating the human perception system is more attuned to changes in green values, and therefore a disproportionate amount of the pixel data needs to be directed towards receiving green image values.
To determine missing pixel values, Cok teaches assigning an unknown pixel T a value derived from contributions from values in an immediately surrounding pixel neighborhood. For example, consider a 3xc3x973 pixel sequence from an image sensor""s green components: 
In Cok, the unknown pixel T is assigned a value expressed as a combination of known, pixels B1-B4. The exact contribution of each B is decided by averaging B1-B4, and then comparing the average against each B value to determine a binary pattern of high-low values based on the comparison. The pattern is looked up in a table which maintains a concordance between particular patterns and classifies the pattern found as either a stripe, corner or edge. The contributions to be ascribed to each neighborhood pixel are then defined as one of three specific formulas for computing T.
A significant problem with this and related techniques, however, is that T is being defined with respect to a very limited context area. Consequently, when analyzing ambiguous input data, or high-detail image data, the guessed value may be sub-optimal. As the resolution of today""s sensors increase, the differences between adjacent PELs is changed. When combined with a consumer""s expectation of a sharper image, and the lower cost of applying complex algorithms, the technique described by Cok is no longer satisfactory.
In one embodiment of the invention, a method is disclosed for determining a value for an unknown pixel T. An immediate neighborhood of pixel values and an extended neighborhood of pixels values are selected. An average pixel value of the immediate neighborhood of pixel values is computed. The average pixel value is compared to the immediate and extended neighborhood pixel values, and a binary pattern determined based on the comparing. The binary pattern is reduced and a set of coefficients identified for the reduced binary pattern.